<!DOCTYPE html>
<html class="client-nojs vector-feature-night-mode-disabled vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-1 vector-feature-appearance-pinned-clientpref-1 vector-sticky-header-enabled" lang="en" dir="ltr"><head>
<meta charset="UTF-8">
<title>Machine-generated data</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link rel="canonical" href="https://en.wikipedia.org/wiki/Machine-generated_data"> <link href="./mw/ext.cite.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.icons.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.search.codex.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/skins.vector.styles.css" rel="stylesheet" type="text/css">
<link href="./mw/user.styles.css" rel="stylesheet" type="text/css">
<meta name="ResourceLoaderDynamicStyles" content="">
<link rel="stylesheet" type="text/css" href="./mw/site.styles.css">
<link rel="stylesheet" type="text/css" href="./mw/noscript.css">
<link rel="stylesheet" type="text/css" href="./footer.css">
<link rel="stylesheet" type="text/css" href="./vector-2022.css">
</head>
<body class="skin--responsive skin-vector skin-vector-search-vue mediawiki ltr sitedir-ltr mw-hide-empty-elt ns-0 ns-subject page-Machine-generated_data rootpage-Machine-generated_data skin-vector-2022 action-view">
<div class="mw-page-container">
<div class="mw-page-container-inner">
<div class="mw-content-container">
<main id="content" class="mw-body">
<header class="mw-body-header vector-page-titlebar">
<h1 id="firstHeading" class="firstHeading mw-first-heading">
<span id="openzim-page-title" class="mw-page-title-main"><span class="mw-page-title-main">Machine-generated data</span></span>
</h1>
</header>
<a id="top"></a>
<div id="bodyContent" class="vector-body ve-init-mw-desktopArticleTarget-targetContainer" aria-labelledby="firstHeading" data-mw-ve-target-container="">
<div id="mw-content-text" class="mw-body-content mw-content-ltr" lang="en" dir="ltr"><div class="mw-content-ltr mw-parser-output" lang="en" dir="ltr"><p><b>Machine-generated data</b> is <a href="Information" title="Information">information</a> automatically generated by a <a href="Computer_process" class="mw-redirect" title="Computer process">computer process</a>, <a href="Computer_application" class="mw-redirect" title="Computer application">application</a>, or other mechanism without the active intervention of a human. While the term dates back over fifty years,<sup id="cite_ref-1" class="reference"><a href="#cite_note-1"><span class="cite-bracket">[</span>1<span class="cite-bracket">]</span></a></sup> there is some current indecision as to the scope of the term. Monash Research's Curt Monash defines it as "data that was produced entirely by machines OR data that is more about observing humans than recording their choices."<sup id="cite_ref-2" class="reference"><a href="#cite_note-2"><span class="cite-bracket">[</span>2<span class="cite-bracket">]</span></a></sup> Meanwhile, Daniel Abadi, CS Professor at <a href="Yale_University" title="Yale University">Yale</a>, proposes a narrower definition, "Machine-generated data is data that is generated as a result of a decision of an independent computational agent or a measurement of an event that is not caused by a human action."<sup id="cite_ref-3" class="reference"><a href="#cite_note-3"><span class="cite-bracket">[</span>3<span class="cite-bracket">]</span></a></sup> Regardless of definition differences, both exclude data manually entered by a person.<sup id="cite_ref-dbms2example_4-0" class="reference"><a href="#cite_note-dbms2example-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> Machine-generated data crosses all <a href="Industry_sector" class="mw-redirect" title="Industry sector">industry sectors</a>. Often and increasingly, humans are unaware their actions are generating the data.<sup id="cite_ref-5" class="reference"><a href="#cite_note-5"><span class="cite-bracket">[</span>5<span class="cite-bracket">]</span></a></sup>
</p>
<meta property="mw:PageProp/toc">
<div class="mw-heading mw-heading2"><h2 id="Relevance">Relevance</h2></div>
<p>Machine-generated data has no single form; rather, the type, format, <a href="Metadata" title="Metadata">metadata</a>, and frequency respond to some particular business purpose. Machines often create it on a defined time schedule or in response to a state change, action, transaction, or other event. Since the event is historical, the data is not prone to be updated or modified. Partly because of this quality, the <a href="United_States" title="United States">U.S.</a> <a href="Court_system" class="mw-redirect" title="Court system">court systems</a> consider machine-generated data as highly reliable.<sup id="cite_ref-6" class="reference"><a href="#cite_note-6"><span class="cite-bracket">[</span>6<span class="cite-bracket">]</span></a></sup>
</p><p>Machine-generated data is the lifeblood of the <a href="Internet_of_Things" class="mw-redirect" title="Internet of Things">Internet of Things</a> (IoT).<sup id="cite_ref-7" class="reference"><a href="#cite_note-7"><span class="cite-bracket">[</span>7<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading2"><h2 id="Growth">Growth</h2></div>
<p>In 2009, <a href="Gartner" title="Gartner">Gartner</a> published that data will grow by 650% over the following five years.<sup id="cite_ref-sciencelogic_8-0" class="reference"><a href="#cite_note-sciencelogic-8"><span class="cite-bracket">[</span>8<span class="cite-bracket">]</span></a></sup> Most of the growth in data is the byproduct of machine-generated data.<sup id="cite_ref-dbms2example_4-1" class="reference"><a href="#cite_note-dbms2example-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> IDC estimated that in 2020, there will be 26 times more connected things than people.<sup id="cite_ref-9" class="reference"><a href="#cite_note-9"><span class="cite-bracket">[</span>9<span class="cite-bracket">]</span></a></sup> Wikibon issued a forecast of $514 billion to be spent on the <a href="Industrial_Internet" class="mw-redirect" title="Industrial Internet">Industrial Internet</a> in 2020.<sup id="cite_ref-10" class="reference"><a href="#cite_note-10"><span class="cite-bracket">[</span>10<span class="cite-bracket">]</span></a></sup>
</p>
<div class="mw-heading mw-heading3"><h3 id="Processing">Processing</h3></div>
<p>Given the fairly static yet voluminous nature of machine-generated data, data owners rely on highly scalable tools to process and analyze the resulting <a href="Dataset" class="mw-redirect" title="Dataset">dataset</a>. Almost all machine-generated data is unstructured but then derived into a common structure.<sup id="cite_ref-dbms2example_4-2" class="reference"><a href="#cite_note-dbms2example-4"><span class="cite-bracket">[</span>4<span class="cite-bracket">]</span></a></sup> Typically, these derived structures contain many <a href="Data_point" class="mw-redirect" title="Data point">data points</a>/columns. With these data points, the challenge lies mostly with analyzing the data. Given high performance requirements along with large data sizes, traditional <a href="Database_index" title="Database index">database indexing</a> and partitioning limits the size and history of the dataset for processing. Alternative approaches exist with <a href="Columnar_database" class="mw-redirect" title="Columnar database">columnar databases</a> as only particular "columns" of the dataset would be accessed during particular analysis.
</p>
<div class="mw-heading mw-heading2"><h2 id="Examples">Examples</h2></div>
<ul><li><a href="Server_log" class="mw-redirect" title="Server log">Web server logs</a><sup id="cite_ref-monashexamples_11-0" class="reference"><a href="#cite_note-monashexamples-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li>
<li><a href="Call_detail_record" title="Call detail record">Call detail records</a><sup id="cite_ref-monashexamples_11-1" class="reference"><a href="#cite_note-monashexamples-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li>
<li><a href="Financial_instrument" title="Financial instrument">Financial instrument</a> trades<sup id="cite_ref-monashexamples_11-2" class="reference"><a href="#cite_note-monashexamples-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li>
<li>Network <a href="Event_log" class="mw-redirect" title="Event log">event logs</a><sup id="cite_ref-monashexamples_11-3" class="reference"><a href="#cite_note-monashexamples-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li>
<li>Logs transmitted from security, network and OS sources to <a href="Security_information_and_event_management" title="Security information and event management">Security information and event management</a> (SIEM) systems</li>
<li><a href="Telemetry" title="Telemetry">Telemetry</a> collected by the government<sup id="cite_ref-monashexamples_11-4" class="reference"><a href="#cite_note-monashexamples-11"><span class="cite-bracket">[</span>11<span class="cite-bracket">]</span></a></sup></li></ul>
<div class="mw-heading mw-heading2"><h2 id="Notes">Notes</h2></div>
<div class="mw-heading mw-heading3"><h3 id="Reference_List">Reference List</h3></div>
<style data-mw-deduplicate="TemplateStyles:r1239543626">
/* start https://en.wikipedia.org/ */
.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}
/* end https://en.wikipedia.org/ */
</style><div class="reflist reflist-columns references-column-width" style="column-width: 30em;">
<ol class="references">
<li id="cite_note-1"><span class="mw-cite-backlink"><b><a href="#cite_ref-1">^</a></b></span> <span class="reference-text"><style data-mw-deduplicate="TemplateStyles:r1238218222">
/* start https://en.wikipedia.org/ */
.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}
/* end https://en.wikipedia.org/ */
</style><cite class="citation book cs1"><a rel="nofollow" class="external text" href="https://books.google.com/books?id=D1qDu4nTmvsC"><i>Control Systems Functions and Programming Approaches: Applications by Dimitris N Chorafas</i></a>. Academic Press. 1966-01-01. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-08-095534-6</bdi>.</cite></span>
</li>
<li id="cite_note-2"><span class="mw-cite-backlink"><b><a href="#cite_ref-2">^</a></b></span> <span class="reference-text">Monash, 12/30/2010</span>
</li>
<li id="cite_note-3"><span class="mw-cite-backlink"><b><a href="#cite_ref-3">^</a></b></span> <span class="reference-text">Abadi</span>
</li>
<li id="cite_note-dbms2example-4"><span class="mw-cite-backlink">^ <a href="#cite_ref-dbms2example_4-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-dbms2example_4-1"><sup><i><b>b</b></i></sup></a> <a href="#cite_ref-dbms2example_4-2"><sup><i><b>c</b></i></sup></a></span> <span class="reference-text">Monash, Three Broad Categories of Data</span>
</li>
<li id="cite_note-5"><span class="mw-cite-backlink"><b><a href="#cite_ref-5">^</a></b></span> <span class="reference-text">Deloach, Machine Generated Data</span>
</li>
<li id="cite_note-6"><span class="mw-cite-backlink"><b><a href="#cite_ref-6">^</a></b></span> <span class="reference-text">Federal Evidence Review, Machine Generated Data was Not Statement and Raised no Hearsay</span>
</li>
<li id="cite_note-7"><span class="mw-cite-backlink"><b><a href="#cite_ref-7">^</a></b></span> <span class="reference-text"><cite id="CITEREFSeth_Grimes2016" class="citation web cs1">Seth Grimes [@SethGrimes] (8 March 2016). <a rel="nofollow" class="external text" href="https://x.com/SethGrimes/status/707286159135784960">"Machine-generated data is the lifeblood of the Internet of Things (#IoT): a key but missing point"</a> (<a href="Tweet_(social_media)" title="Tweet (social media)">Tweet</a>) – via <a href="Twitter" title="Twitter">Twitter</a>.</cite></span>
</li>
<li id="cite_note-sciencelogic-8"><span class="mw-cite-backlink"><b><a href="#cite_ref-sciencelogic_8-0">^</a></b></span> <span class="reference-text">ScienceLogic</span>
</li>
<li id="cite_note-9"><span class="mw-cite-backlink"><b><a href="#cite_ref-9">^</a></b></span> <span class="reference-text"><a rel="nofollow" class="external autonumber" href="http://chucksblog.emc.com/chucks_blog/2012/12/charting-the-digital-universe-idcs-6th-annual-study.html">[1]</a>, Chuck's Blog</span>
</li>
<li id="cite_note-10"><span class="mw-cite-backlink"><b><a href="#cite_ref-10">^</a></b></span> <span class="reference-text"><a rel="nofollow" class="external autonumber" href="http://wikibon.org/wiki/v/Defining_and_Sizing_the_Industrial_Internet">[2]</a>, Wikibon</span>
</li>
<li id="cite_note-monashexamples-11"><span class="mw-cite-backlink">^ <a href="#cite_ref-monashexamples_11-0"><sup><i><b>a</b></i></sup></a> <a href="#cite_ref-monashexamples_11-1"><sup><i><b>b</b></i></sup></a> <a href="#cite_ref-monashexamples_11-2"><sup><i><b>c</b></i></sup></a> <a href="#cite_ref-monashexamples_11-3"><sup><i><b>d</b></i></sup></a> <a href="#cite_ref-monashexamples_11-4"><sup><i><b>e</b></i></sup></a></span> <span class="reference-text">Monash, Examples of Machine Generated Data</span>
</li>
</ol></div>
<div class="mw-heading mw-heading3"><h3 id="Bibliography">Bibliography</h3></div>
<style data-mw-deduplicate="TemplateStyles:r1239549316">
/* start https://en.wikipedia.org/ */
.mw-parser-output .refbegin{margin-bottom:0.5em}.mw-parser-output .refbegin-hanging-indents>ul{margin-left:0}.mw-parser-output .refbegin-hanging-indents>ul>li{margin-left:0;padding-left:3.2em;text-indent:-3.2em}.mw-parser-output .refbegin-hanging-indents ul,.mw-parser-output .refbegin-hanging-indents ul li{list-style:none}@media(max-width:720px){.mw-parser-output .refbegin-hanging-indents>ul>li{padding-left:1.6em;text-indent:-1.6em}}.mw-parser-output .refbegin-columns{margin-top:0.3em}.mw-parser-output .refbegin-columns ul{margin-top:0}.mw-parser-output .refbegin-columns li{page-break-inside:avoid;break-inside:avoid-column}@media screen{.mw-parser-output .refbegin{font-size:90%}}
/* end https://en.wikipedia.org/ */
</style><div class="refbegin" style="">
<ul><li><cite id="CITEREFAbadi" class="citation news cs1">Abadi, Daniel. <a rel="nofollow" class="external text" href="http://dbmsmusings.blogspot.com/2010/12/machine-vs-human-generated-data.html">"Machine vs. Human generated data"</a>. BlogSpot.</cite></li>
<li><cite id="CITEREFDeloach" class="citation news cs1">Deloach, Don. <a rel="nofollow" class="external text" href="http://www.infobright.com/Blog/Entry/machine_generated_data/">"Machine Generated Data"</a>. Infobright, Inc.</cite></li>
<li><cite id="CITEREFFederal_Evidence_Review" class="citation news cs1">Federal Evidence Review. <a rel="nofollow" class="external text" href="http://federalevidence.com/blog/2008/december/machine-generated-data-was-not-statement-and-raised-no-hearsay-or-confrontation-c">"Machine Generated Data Was Not Statement and Raised no Hearsay or Confrontation"</a>.</cite></li>
<li><cite id="CITEREFMonash" class="citation news cs1">Monash, Curt. <a rel="nofollow" class="external text" href="http://www.dbms2.com/2010/01/17/three-broad-categories-of-data/">"Three Broad Categories of Data"</a>. Monash Research.</cite></li>
<li><cite id="CITEREFMonash" class="citation news cs1">Monash, Curt. <a rel="nofollow" class="external text" href="http://www.dbms2.com/2010/04/08/machine-generated-data-example/">"Examples of Machine Generated Data"</a>. Monash Research.</cite></li>
<li><cite id="CITEREFMonash" class="citation news cs1">Monash, Curt. <a rel="nofollow" class="external text" href="http://www.dbms2.com/2010/12/30/examples-and-definition-of-machine-generated-data/">"Examples and definition of machine-generated data"</a>. Monash Research.</cite></li>
<li><cite id="CITEREFScience_Logic" class="citation news cs1">Science Logic. <a rel="nofollow" class="external text" href="http://blog.sciencelogic.com/gartner-ten-technologies-to-watch/06/2009/">"Gartner Ten Technologies to Watch"</a>.</cite></li></ul>
</div></div><!--htdig_noindex--><div><div class="zim-footer">
This article is issued from <a class="external text" title="Last edited on 2025-01-25" href="https://en.wikipedia.org/wiki/?title=Machine-generated_data&oldid=1271661280">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.
</div>
</div><!--/htdig_noindex--></div>
</div>
</main>
</div>
</div>
</div>
</body></html>